Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

optimize ai cache #1626

Merged
merged 2 commits into from
Dec 27, 2024
Merged

optimize ai cache #1626

merged 2 commits into from
Dec 27, 2024

Conversation

johnlanni
Copy link
Collaborator

@johnlanni johnlanni commented Dec 26, 2024

Ⅰ. Describe what this PR did

According to whatwag's rules, any of CRLF/CR/LF can be used as the end delimiter for an SSE message.

LLM models compatible with xinference framework deployment

Ⅱ. Does this pull request fix one issue?

fixes #1624

Ⅲ. Why don't you add test cases (unit test/integration test)?

Ⅳ. Describe how to verify it

Ⅴ. Special notes for reviews

@codecov-commenter
Copy link

codecov-commenter commented Dec 26, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 43.50%. Comparing base (ef31e09) to head (09102ce).
Report is 244 commits behind head on main.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #1626      +/-   ##
==========================================
+ Coverage   35.91%   43.50%   +7.59%     
==========================================
  Files          69       76       +7     
  Lines       11576    12325     +749     
==========================================
+ Hits         4157     5362    +1205     
+ Misses       7104     6627     -477     
- Partials      315      336      +21     

see 69 files with indirect coverage changes

Copy link
Collaborator

@rinfx rinfx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@CH3CHO CH3CHO merged commit 6dc4d43 into alibaba:main Dec 27, 2024
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

AI-Cache插件针对流式请求不缓存的问题
4 participants